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Abstract 

Motion correction is the first in a pipeline of algorithms to analyze calcium imag¬ 
ing videos and extract biologically relevant information, for example the network 
structure of the neurons therein. Fast motion correction would be especially critical 
for closed-loop activity triggered stimulation experiments, where accurate detection 
and targeting of specific cells in necessary. Our algorithm uses a Fourier-transform 
approach, and its efficiency derives from a combination of judicious downsampling 
and the accelerated computation of many L2 norms using dynamic programming 
and two-dimensional, fft-accelerated convolutions. Its accuracy is comparable to 
that of established community-used algorithms, and it is more stable to large trans¬ 
lational motions. It is programmed in Java and is compatible with ImageJ. 


1 Introduction 

Calcium imaging, first used to measure the activity of neurons in the early 1990s [fOj, has 
been successfully applied throughout the nervous system. It allows us to see the behavior 
of neurons in awake behaving mice, using either chemical or genetic calcium indicators, 
with confocal microscopy, two-photon microscopy, or wearable imaging devices jJ] As 
a result, it is an increasingly useful tool for identifying the neural substrate of mouse 
behaviors. However, calcium imaging videos have difficult noise properties, including 
white noise and motion artifacts which must be corrected in a preprocessing step before 
proper analysis can be undertaken. 

Motion correction is the first step in the analysis of calcium images. After they 
are motion-corrected, ROIs are identified, and time-activity graphs are made from each 
ROI. If the motion-correction is low-quality, then the time-activity graphs suffer, and 
the reconstructed networks may have errors. For real time closed loop operation, if the 
motion correction is slow, it cannot be done while the mouse is in the microscope, and 
the experiment fails. 

TurboReg [9] is a commonly used algorithm to do motion correction. It uses a down¬ 
sampling strategy, which is prerequisite for speed, and it uses a template image, which 
is necessary for accuracy. We have developed a similar method, called moco (MOtion 
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COrrector), which adopted both strategies, since correcting one image against the next 
in the stack results in unacceptable roundoff errors. Other approaches use HMMs n. 0 
or other techniques a, a, 0. ra. B- a is the only one similar mathematically to, and 
may be slightly faster than, moco, but it has accuracy problems (see Figure 2). 

moco uses downsampling and a template image, and it can be called from ImageJ. 
However, it is faster than TurboReg [9j at translation-based motion correction because 
it uses dynamic programming and two-dimensional fft-acceleration of two-dimensional 
convolutions. [I] also uses the fft approach but uses a different objective function that 
does not require dynamic programming; we believe that our approach is more robust to 
corrupted data, see Figure 2. Image Stabilizer is as fast for small images, but is very 
slow for standard-size images. Running on our own datasets, moco appears faster than 
all approaches compatible with ImageJ. 

moco corrects every image in the video by comparing every possible translation of 
it with the template image, and chooses the one which minimizes the Lo norm of the 
difference between the images in the overlapping region, D, divided by the area of D. 
The fact that it is so thorough makes it robust to long translations in the data. More 
complicated non-translation image warps are usually unnecessary for fixing calcium im¬ 
ages, which suffer from spurious translations, which moco corrects, and spurious motion 
in the Z-direction, which is very hard to correct. Our approach also uses cache-aware 
upsampling: when an image is aligned with the template in the downsampled space, it 
must be jittered when it is upsampled to see which jitter best aligns with the upsampled 
template. We do this in such a way that data that is used recently is reused immedi¬ 
ately, making the implementation faster. Hence, moco is an efficient motion correction 
of calcium images, and is likely to become a useful tool for processing calcium imaging 
movies. 


2 Mathematical Development 

Let ctjj, for i = 1 ,,m and j = 1,..., n be an image in the stack. We assume a is 
downsampled if it is larger than 256 x 256. Let b, tJ be the template image against which 
to align a. We want to pick (s,t) such that rnax(|sj, \t\) < w, where w is input by the 
user, and 

Ut = Area (D„) £ ( “‘ + « + * “ M 2 

is minimal, where D s j is the set of ordered pairs of integers ( i',j ') such that 1 < i' < m, 
1 < j 1 < n, 1 < i’ + s < m, and 1 < j 1 + t < n. If we do this for every image a 
in the stack, we have then motion corrected the video, and we are done, up to a short 
upsampling step. To upsample, multiply the optimal (s, t ) by 2 and do a local search to 
minimize f s j on the hner grid. Now, 

Area (D S) t)f s ,t = ^ ^ a i+s,j+t J~ ^ ^ ^i,j ~ ^ ^ cii+ S j+tbij- 

(i,j)£D s , t ( i,j)£D S} t 
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The first two sums can be computed via dynamic programming. Let’s consider a when 
s and t are negative. Let 

9s,t = ^2 a i+s,i+f 

(i,j)eD s , t 

We have that 

_ | ,2 
9s,t — 9s-i,t + 9s,t -1 — 9s-i,t -1 + a m+sn+t . 

Hence, the first two sums can be computed for all (s, t) in 0{inn ) time, which is unaffected 
by a constant amount of downsampling. It suffices to compute for all (s, t) such that 
max(|s|, \t\) < w, 

h s ,t ^ ^ (^i+s,j+tbi,j • 

Let b be b rotated 180 degrees. Using MATLAB notation, let 

a = fft2([[a, zeros(m, w)]; zeros (w, n + w)]), 
b = fft2([[6, zeros(m, w)]; zeros(w, n + w)]). 

Commas denote horizontal concatenation, semicolons denote vertical concatenation, and 
zeros(a:, y) is an x x y matrix of zeros. For equally sized matrices X , Y, let Z — X © Y 
mean Z, L j = XijY hJ . Then 

ifft2(a 0 b) 

is a rearrangement of h. Since fft2’s are fast, that means h can be computed for all 
(s,t) in 0(rim log(nm)) time. Hence, after upsampling, the entire video can be aligned 
in O(mnT log (mn)) time, where T is the number of slides in the video. 

After (s,t) are chosen to minimize f Si t, they are multiplied by two multiple times 
to upsample. Every time they are multiplied by 2, f-is+u.n+v are computed for u,v G 
{0, —1,1} to see which u and v are minimal. These nine evaluations of / are done with 
a cache-aware algorithm for speed. 

3 Results 

We compare moco in speed to TurboReg [9J on its translation mode, using both the “fast” 
and “accurate” settings. We also compare it to Image Stabilizer using its default settings 
[6] (it can be made faster by changing the settings but the accuracy is poor). We use 
several real calcium imaging videos, which we say are m x n x T if they contain T slides 
of size m x n. If the images are larger than 256 x 256, we downsample once, otherwise, 
we do not downsample. We have found that dowsampling 3 and 4 times causes severe 
errors so we avoid those settings. In addition, we have compared moco to TurboReg on 
synthetic images with severe translational motion artifacts and have found that moco is 
slightly more accurate. All times are in seconds. The template used for every video is the 
first image in the video for both moco and TurboReg. moco uses a maximum translation 
width of min(m, n)/3 in both the i and j directions. 
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Size 

moco 

TurboReg 

TurboReg (slow) 

Image Stabilizer 

512 x 512 x 1500 

66s 

110s 

242s 

304s 

512 x 512 x 2000 

90s 

170s 

298s 

464s 

512 x 512 x 6984 

288s 

632s 

1303s 

2277s 

416 x 460 x 1000 

35s 

71s 

132s 

41s 

256 x 256 x 2028 

84s 

121s 

154s 

34s 


As is clear from the table, moco is faster than its most used current method, TurboReg. 
It may be marginally slower than [3], but Figure 2 proves that a code we have created to 
have similar results to [4] is inaccurate. Figure 1 shows the first two images of a corrupted 
video on the first row. moco corrections are on the second row. It is clear that moco can 
fix the image, even though the problems with it are severe. Figure 2 shows the mean 
image from a corrupted video, and the mean image of moco and TurboReg corrections, 
as well as the correction from our MATLAB version of the [4J algorithm. 
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l.a. 2.a. 



l.b. 


2.b. 


Figure 1: Images are 317.44/im x 317.44/im. l.a. and 2.a. are the first two images of a 
long, badly corrupted video submitted to moco. l.b. and 2.b. are the first two corrected 
images. One can see that l.a. and l.b. are the same, since l.a. is used as the template 
image. However, 2.b. is moved, so that if it is moved to the left to overlap l.b., it matches 
it almost perfectly except where it is black. 
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l.c. 2.c. 


Figure 2: Images are 317.44/rm x 317.44/rm. l.a. is the mean of every image in a badly 
corrupted video. 2.a. is the mean of the corrected video using our implementation of the 
[4] approach, l.b. is the mean of the corrected video using moco. 2.b. is the mean of the 
corrected video using TurboReg (accurate mode), [9] l.c. is the mean of the corrected 
video using TurboReg (accurate mode). 2.c. is the mean of the corrected video using 
Image Stabilizer. We see that moco and TurboReg have superior performance. 
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